Method and system of detecting foreign materials within an agricultural product stream

ABSTRACT

Methods and systems for detecting foreign material within a product stream in real-time, involve illuminating a portion of the agricultural product stream with light spanning a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; scanning a line of the illuminated agricultural product stream to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; processing the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line; comparing the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the agricultural product stream.

FIELD OF THE INVENTION

The present disclosure relates to a method and system of detectingforeign material within an agricultural product stream. Disclosed hereinis an on-line system and method for the detection and separation ofunwanted materials and/or foreign matter from a product stream byemploying hyperspectral imaging and advanced data processing andclassification algorithms. Embodiments disclosed herein can be practicedwith tobacco as well as other agricultural products, including tea,grapes, coffee, vegetables, fruit, nuts, breads, cereals, and otherplant or animal parts.

BACKGROUND

Tobacco delivered for processing may occasionally contain foreign mattersuch as pieces of a container in which it is shipped and/or stored, bitsof string and paper, foam, cardboard, foil, and/or other items. There isa need for methods and systems to remove foreign, non-tobacco relatedmaterials from tobacco prior to its incorporation into tobacco products.Further, there is a need for such methods and systems to be capable ofconducting on-line detection of non-tobacco related materials, inreal-time, without requirements for extensive computing resources orlimited belt speeds.

SUMMARY OF SELECTED FEATURES

Disclosed herein is a method of detecting foreign material within anagricultural product stream in real-time. The method comprisesilluminating a portion of the agricultural product stream with lightspanning a wavelength range including or within near-infrared and/orshortwave infrared wavelengths. A line of the illuminated agriculturalproduct stream is scanned to acquire a hyperspectral image of the linewherein the hyperspectral image of the line has a width of a singlepixel. The hyperspectral image of the scanned line is processed toobtain spectrum data for one or more pixels of the hyperspectral imageof the scanned line. The obtained spectrum data of the one or morepixels is compared to predetermined spectrum data to determine whetherthe obtained spectrum data is indicative of foreign material within thescanned line of the agricultural product stream.

Also disclosed herein is a system for detecting foreign material withinan agricultural product stream in real-time. The system comprises anillumination device configured to illuminate a portion of theagricultural product stream at a wavelength range including or withinnear-infrared and/or shortwave infrared wavelengths. A hyperspectralimaging device is configured to scan a line of the illuminatedagricultural product stream to acquire a hyperspectral image of the linewherein the hyperspectral image of the line has a width of a singlepixel. A processor is configured to: process the hyperspectral image ofthe scanned line to obtain spectrum data for one or more pixels of thehyperspectral image of the scanned line, and compare the obtainedspectrum data of the one or more pixels to predetermined spectrum datato determine whether the obtained spectrum data is indicative of foreignmaterial within the scanned line of the agricultural product stream.

Further disclosed herein is a method of detecting foreign materialwithin stationary agricultural product in real-time. The methodcomprises illuminating a portion of the stationary agricultural productwith light spanning a wavelength range including or within near-infraredand/or shortwave infrared wavelengths. A hyperspectral imaging device ismoved over a portion of the stationary agricultural product while thehyperspectral imagining device simultaneously scans a line of theilluminated stationary agricultural product to acquire a hyperspectralimage of the line. The hyperspectral image of the line has a width of asingle pixel. The hyperspectral image of the scanned line is processedto obtain spectrum data for one or more pixels of the hyperspectralimage of the scanned line. The obtained spectrum data of the one or morepixels is compared to predetermined spectrum data to determine whetherthe obtained spectrum data is indicative of foreign material within thescanned line of the stationary agricultural product.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic representation of a detection and separationsystem which may be implemented in connection with the presentdisclosure.

FIG. 2 depicts a detailed embodiment of a detection and separationsystem which may be implemented in connection with the presentdisclosure.

FIG. 3 presents, in a flow diagram, a testing or detection phase of amethod for detection of foreign matter in a product stream (e.g., anagricultural product stream).

FIG. 4 presents, in a flow diagram, a training phase which may beimplemented in connection with a method for detection of foreign matterin a product stream.

FIG. 5 sets forth a graph of sample spectra acquired by a hyperspectralnear-infrared imaging instrument.

FIG. 6 depicts use of the systems and methods disclosed herein fordetection of non-product related foam within a product stream of tobaccoproduct.

DETAILED DESCRIPTION

Disclosed herein are methods and systems for the detection and/orremoval of foreign matter from a product stream, such as an agriculturalproduct stream. By way of illustration, the systems and methodsdisclosed herein will be described for application during tobaccoprocessing, but a system as disclosed herein could be applied during theprocessing of other products, such as other agricultural products. Thedetection and separation systems disclosed herein can be used in manyprocesses and for consumer products which are susceptible to thepresence of unwanted materials during a manufacturing process, such as,for example, in the growing, collection, processing and/or packaging ofpackaged consumer goods, such as food products, beverages, tipped andnon-tipped cigars, cigarillos, snus and other smokeless tobaccoproducts, smoking articles, electronic cigarettes, distilled products,pharmaceuticals, frozen foods and other comestibles, and the like.Further applications could include clothing, furniture, lumber, or anyother manufactured or packaged product wherein an absence of non-productor foreign material is desired.

Tobacco may first be harvested at a farm, which in the case of tobaccofor use in cigarette manufacturing or the production of moist smokelesstobacco (MST), may be harvested at least in part by machinery. Leaves oftobacco may be baled and received at a receiving station from the farm.Notably, the opportunity exists for the tobacco bale to come intocontact with lubricated machinery at a receiving station as well as withother non-product related materials (i.e., foreign materials). The baledtobacco may be transferred to a stemmery wherein large stems may beremoved by machines to produce de-stemmed tobacco. The de-stemmedtobacco can be packed into bales which are then stored for a suitabletime period of up to several years. De-stemmed tobacco is thentransferred to manufacturing centers, wherein various types of tobaccomay be blended according to a predetermined recipe. The blended tobaccomay be treated by adding various flavorants to provide a cased tobacco,which may be cut or shredded. Various other types of tobacco can beadded to the blended tobacco including puffed tobacco, reconstitutedtobacco, reclaimed tobacco, and the like, to provide a final productblend. The final product blend may be then fed to a make/pack machine,which includes a continuous tobacco rod making apparatus that makestobacco rods from the final product blend. The tobacco rod may then becut, optionally tipped, and packed, typically through the use ofhigh-speed machinery.

As may be appreciated from the above description, in tobacco processing,tobacco comes into contact with machinery at many different points inthe overall process, such as machinery used during the growing andharvesting operations on the farm, handling equipment at the receivingstation or auction house, machinery in the stemmery, on conveyors,conditioners, cutters, silos, and other equipment at a manufacturingcenter. Additionally, as may be appreciated from the above description,various points exist during the process of tobacco product manufacturingin which non-product related material may be introduced to the productstream. Such non-product related material may include foreign mattersuch as foil, cellophane, warehouse tags, cardboard, foam, paper, or oilor other lubricant containing material.

Due to the inadvertent introduction of non-product material into aproduct stream during tobacco processing, there is a need for methodsand systems that identify and remove non-product material from suchproduct streams. The embodiments disclosed herein are generally focusedon the manufacturing or processing of tobacco, blend components orsamples, and the automated monitoring thereof by use of hyperspectralimaging technology. It should be understood that the embodimentsdisclosed herein could be applied to other domains encompassing, e.g.,the manufacturing or processing of tea, fruits, during the production offruit juices, grapes for the production of wines, as well as otheragricultural products.

Hyperspectral imaging can be thought of as a combination of spectroscopyand imaging. In spectroscopy a spectra is collected at a single point.Spectra consist of a continuum of values that correspond to measurementsat different wavelengths of light, which contain information about thechemical composition and material properties of a sample from whichspectra are collected. In contrast, traditional cameras collect dataincluding thousands of points. Data from each point or pixel containsone value (black and white image) or three values for a color image,corresponding to colors, red, green, and blue. Hyperspectral camerascombine the spectral resolution of spectroscopy and the spatialresolution of cameras. They can create images with, e.g., hundreds orthousands of pixels, that contain an array of values corresponding tolight measurements at different wavelengths. Thus, the data at eachpixel of a hyperspectral image includes a spectrum of wavelengths oflight.

Hyperspectral imaging and analysis involves the scanning of an objectwhile the object is exposed to electromagnetic radiation (e.g., in theform of light). When the object is illuminated, the electromagneticradiation is affected by one or more of the physical, chemical, and/orbiological species or components from which the object is comprised, byany of, or a combination of electromagnetic radiation absorption,diffusion, reflection, diffraction, scattering, and/or transmissionmechanisms. Moreover, an object whose composition includes organicchemical species or components, ordinarily exhibits some degree offluorescent and/or phosphorescent properties, when illuminated by sometype of electromagnetic radiation or light, such as ultra-violet (UV),visible (VIS), or infrared (IR), types of light. The affectedelectromagnetic radiation, in the form of diffused, reflected,diffracted, scattered, and/or transmitted or emitted electromagneticradiation of the object is directly and uniquely related to thephysical, chemical, and/or biological properties of the object, ingeneral, and of the chemical species or components making up the object,in particular. Thus, the data which may be acquired by a hyperspectralimaging device associated with a scanned object represents uniquecharacteristics associated with the material of the object. The acquireddata may be used to analyze and/or characterize the object forapplications such as those described herein involving the identificationand/or removal of foreign matter from a product or product stream.

By way of example and not limitation, techniques involving hyperspectralimaging and analysis and the implementation thereof in various stage ofagricultural product processing is described in commonly-assigned U.S.application Ser. No. 14/084,213 to Deevi et al. (US PG-Pub. No.2014/0137877); Ser. No. 14/443,986 to Dante et al. (US PG-Pub. No.2015/0347815); Ser. No. 14/443,990 to Dante et al. (US PG-Pub. No.2015/0283586); and Ser. No. 14/443,995 to Deevi et al. (US PG-Pub. No.2015/0289557), the complete disclosures of which are expresslyincorporated herein by reference in their entireties.

Embodiments of methods of detecting foreign material within agriculturalproducts, such as tobacco products, disclosed herein may include thesteps of: illuminating a portion of the agricultural product stream withlight spanning a wavelength range including or within near-infraredand/or shortwave infrared wavelengths; scanning a line of theilluminated agricultural product stream to acquire a hyperspectral imageof the line wherein the hyperspectral image of the line has a width of asingle pixel; processing the hyperspectral image of the scanned line toobtain spectrum data for one or more pixels of the hyperspectral imageof the scanned line; and comparing the obtained spectrum data of the oneor more pixels to predetermined spectrum data to determine whether theobtained spectrum data is indicative of foreign material within thescanned line of the agricultural product stream.

In some embodiments, the method may further comprise a step of:determining, based upon the comparing step, the location of foreignmaterial in the product stream and/or a step of determining, based uponthe comparing step, that the scanned line of the agricultural productstream does not contain foreign material.

The agricultural product may be tobacco and may comprise tobacco leaves,shredded tobacco or other various forms of tobacco disclosed herein. Theforeign material detected by the methods disclosed herein may comprise,e.g., one or more of: a foam, a cardboard, a plastic, a foil, alubricant, or a paper. The predetermined spectrum data may be acalculated value based upon spectral data associated with, e.g.,tobacco, foam, cardboard, plastic, foil, lubricant or paper.

The methods and systems disclosed herein may be implemented inconnection with moving product streams (e.g., of tobacco or otheragricultural product). In an embodiment, product may be positioned on amovable device that is operable to move the product so as to form amoving product stream. For example, product can be disposed on an uppersurface of a conveyor belt such that when the conveyor belt is driven,the moving product stream moves past a hyperspectral imaging device (orinstrument) that is operable to scan a portion of the moving productstream such that foreign matter in the moving product stream may bedetected and subsequently removed. In an embodiment, the product streammay be continuously moving on the conveyor belt at speeds of up to 80 to100 feet per minute.

In an embodiment, the hyperspectral imaging device is positioned in alocation that enables a portion of the moving product stream to bescanned. The hyperspectral imaging device preferably operates in a rangeof about 900 nm to 1700 nm or about 900 nm to 2500 nm. In an embodiment,the hyperspectral imaging device can be a high speed camera having aframe rate of up to about 340 frames per second or greater. In anembodiment, hyperspectral images may be acquired with a line-scanhyperspectral imaging device wherein respective lines of the movingproduct stream are scanned by the hyperspectral imaging device as theproduct moves past the hyperspectral imaging device. In an embodiment,the portion of the moving agricultural product stream that thehyperspectral imaging device scans is a line of the product stream thatis perpendicular to a direction in which the agricultural product streamis moving (e.g., a hyperspectral imaging device may scan a portion of anagricultural product stream perpendicular to the direction in which aconveyor belt is conveying the agricultural product). In an alternativeembodiment, the hyperspectral imaging device can be a movablehyperspectral imaging device that is operable to scan stationary productas the hyperspectral imaging device moves across a portion of theproduct. In this embodiment, an illumination source can be configured tomove with the hyperspectral imaging device.

In an embodiment, a moving product stream (i.e., product on a conveyorbelt) is scanned line-by-line as the product stream passes thehyperspectral imaging device. The hyperspectral imaging devicepreferably captures one or more images of the product wherein each imagecaptured by the hyperspectral imaging device is representative of ascanned line (i.e., a single row of pixels). Preferably, each image (orone or more pixels of an image) is processed in real-time such thatnon-product material and/or foreign material may be detected. Byscanning and processing respective single lines that are captured insuccession, and/or one or more pixels of the respective single linescaptured in succession, redundant data gathering can be omitted. In thismanner, processing time of the images can be reduced, and processingpower can be conserved. Preferably, the methods for detection and/oranalysis and/or removal of non-product material and/or foreign matterdisclosed herein may be performed within systems having a continuouslymoving product stream, such as continuously moving product streamtraveling on a conveyor belt at speeds of up to about 100 feet perminute, and preferably at speeds of about 80 to 100 feet per minute. Insuch embodiments, the detection of foreign matter and/or non-productmaterial may be performed with resolutions, e.g., as low as 1 mm/pixel.In some embodiments, the hyperspectral imaging device may have aspectral resolution of around 5 nm. In some embodiments thehyperspectral imaging device may have a spatial resolution of around0.25 inches. In some embodiments, the hyperspectral imaging device maybe characterized by a frame rate of about 340 frames per second.

In some embodiments, the hyperspectral imaging device may record imageswith up to 67 spectral bands. In some embodiments, the hyperspectralimaging device may record images with up to 100 spectral bands. In someembodiments, the hyperspectral imaging device may record images with upto 150 spectral bands. In some embodiments, the hyperspectral imagingdevice may record images with more than 150 spectral bands. In someembodiments, the hyperspectral imaging device may record images with upto 320 spectral bands. In some embodiments, the hyperspectral imagingdevice may record images with up to 525 spectral bands. In someembodiments, the hyperspectral imaging device may record images withgreater than 525 spectral bands.

In an embodiment, the hyperspectral imaging device implemented in thescanning of product may be aberration-corrected to account for and/orreduce or eliminate noise. Such aberration-correction may beaccomplished via, e.g., systems and methods similar to those describedin U.S. Pat. No. 6,522,404 to Mikes et al. or other methods and systemswhich will be apparent to those skilled in the art, or disclosed herein.

In some embodiments of methods as disclosed herein, a scanning step mayinclude sensing a spectrum of light reflected, scattered, or emittedfrom product, such as a moving stream of the agricultural product, withat least one sensor wherein the scanning step is performed by aline-scan hyperspectral imaging device that includes the at least onesensor. In an embodiment, the at least one sensor is an Indium GalliumArsenide (InGaAs) sensor. In an embodiment, the at least one sensor isoperable to sense light in a wavelength range of about 900 nm to 1700 nmor about 900 nm to 2500 nm.

In some embodiments a pixel of a scanned image may have dimensions ofaround, e.g., 1.5 mm×1.5 mm. In some embodiments, a conveyor belt uponwhich product is conveyed may have a width of around 18 inches. In suchembodiments, the hyperspectral imaging device may be configured to havea field of view of at least 18 inches, which is preferably directedperpendicularly to the direction in which the conveyor belt travels.

In some embodiments, multiple lines of product stream may be scannedindividually, in a consecutive manner, as the product passes thehyperspectral imaging device. In some embodiments, each line of productstream is scanned and processed separately (e.g., multiple scanned linesare not processed to construct a hyperspectral image cube, etc.). Eachscanned line may comprise a single row of pixels. The scanned lineimages may be acquired and processed as they are obtained (e.g., inreal-time). For systems and methods disclosed herein, multiple scans ofa single line of product are not required; however, multiple views orimages of a single line of product may be obtained (i.e., scanned) basedupon operator requirements. In some embodiments, each scanned line, orone or more pixels of a respective scanned line, may be separatelyanalyzed to determine whether foreign material is detected in theportion of product material scanned by the hyperspectral imaging device.In some embodiments, a signal may be generated in response to detectedforeign material. In some embodiments, detected foreign material can bemapped when the foreign material is detected in two or moreconsecutively scanned lines. In some embodiments, each consecutivelyscanned line, or one or more pixels thereof, may be analyzed todetermine whether foreign material is detected wherein a signal thatindicates a false positive may be generated when foreign material isdetected in only a single scanned line. In an embodiment, this falsepositive signal may be overridden, such as when an analysis of thesingle scanned line meets or exceeds a predetermined threshold, forexample, a predetermined threshold for a known foreign material.

In some embodiments, the multiple lines (e.g., portions) of productstream are scanned in a spatially consecutive manner. In someembodiments, a product stream may be scanned so that spatial gaps existbetween consecutively scanned lines of product stream. For instance, ahyperspectral imaging device may be configured to scan a first line of amoving product stream and a second line of the moving product streamwherein a leading edge of the second scanned line (in the direction inwhich the product is traveling) is separated from a trailing edge of thefirst scanned line by a distance of about 15 mm or less, 10 mm or less,5 mm or less 2 mm or less, or 1.5 mm or less.

In some embodiments, a step of processing the hyperspectral image of thescanned line to obtain spectrum data for the scanned line and/or one ormore pixels thereof may comprise calibrating the scanned image (e.g., toremove dark values) and normalizing the image. In some embodiments, astep of processing the hyperspectral image of the scanned line to obtainspectrum data for the scanned line and/or one or more pixels thereof maycomprise smoothing the images (e.g. to reduce or eliminate noise) usingmedian filtering. For instance, smoothing can be performed byimplementing the following equation to smooth the image: y[m,n]=median{x [i, j], (i, j)εw), w represents neighborhood defined by the user,centered around location [m,n] in the image.

In some embodiments, a step of processing the hyperspectral image of thescanned line to obtain spectrum data for the scanned line and/or one ormore pixels thereof may comprise calculating the Euclidean distanceassociated with one or more pixels of the scanned line. The Euclideandistance may be calculated by the following formula: d(p,q)=√{squareroot over (Σ_(i=1) ^(n)(q_(i)−p_(i))²)}, wherein p=(p₁, p₂ . . . ,p_(n)) and q=(q₁, q₂ . . . , q_(n)) and p and q are two points inEuclidean n-space. In some embodiments, other classification techniquesmay be used such as Mahalanobis distance measure, Bhattacharya distancemeasure, spectral angle distance measure (SAM), maximum entropyclassifier, curve fitting based on regression analysis, support vectormachine (SVM), clustering techniques including k-means clustering,nearest neighbor clustering, or techniques based on neural networkclassifiers. In an embodiment, a combination of various data reductionand analysis techniques like principal component analysis, orthogonaldecomposition, spectral mixture resolution, etc. could be used as afirst step to reduce the dimensionality of the data and extract relevantfeatures which could be used by a classifier. Preferably, Euclideandistance is implemented during processing to simplify processing of thescanned lines and/or one or more pixels of a scanned line. The Euclideandistance quickly provides calculated spectrum values that may becompared to predetermined spectrum data or threshold values such thatthe presence of foreign material may be accurately detected.

In some embodiments, the predetermined spectrum data may be determinedby virtue of a data library, storing predetermined spectrum dataassociated with known materials or classes of materials (e.g.,particular contaminants such as yellow foam, a specific type oflubricant, etc.; tobacco leaves; 95% pure tobacco leaves; etc.). In someembodiments, the predetermined spectrum data used for detecting foreignmaterial is a threshold value for a material or class of materials. Forinstance, the spectrum data may correspond to a threshold valuecalculated for a given class of known foreign material, such as a foam,a foil, a paper, etc. (e.g., a contaminant known to commonly appearwithin a given product). As another example, the spectrum data maycorrespond to a threshold value calculated for a class of productmaterial or a product material (e.g., uncontaminated tobacco leaves, 95%uncontaminated tobacco leaves, etc.). In some embodiments, thepredetermined spectrum data or threshold value implemented according tomethods and systems disclosed herein for determining whether product ora moving product stream contains contaminated product, and/or foreignmaterial, may be determined by a process such as the following. Theproduct material, class of product materials, known foreign materialand/or class of foreign materials may be illuminated with near-infraredlight spanning a particular wavelength (e.g., near-infrared, shortwaveinfrared, etc.). Such a wavelength may be similar to the wavelength oflight used to illuminate product or a product stream in a method fordetecting product material and/or foreign matter within that product orproduct stream. In some embodiments, the product and other knownmaterials may be illuminated by light spanning a wavelength of 900 nm to2500 nm, 900 nm to 1700 nm, or a wavelength within or comprising thoseranges. In some preferred embodiments, the illuminating source may be atungsten halogen light source.

The process may further involve separately scanning the productmaterial, class of product materials, respective known foreignmaterials, and/or respective classes of known foreign materials toacquire hyperspectral reflectance image data for at least one of therespective product materials, class of product materials, known foreignmaterials, or class of known foreign materials. The process may furtherinvolve calculating, for at least one of the respective productmaterials, the class of product materials, the known foreign material,or the class of known foreign materials, a mean value of reflectanceintensity, and based upon the one or more calculated mean values ofreflectance intensity, determining the threshold value associated withat least one of the respective product material, the class of productmaterials, the known foreign material, or the class of known foreignmaterials.

To arrive at a threshold, hyperspectral images (e.g., hyperspectralreflectance images) of known materials may be acquired separately with ahyperspectral imaging device operating at a wavelength range similar to,or within, that of the light source illuminating the known materials. Animage for a known material (e.g., yellow foam) may be acquiredseparately from an image for a second known material (e.g., pure tobaccoleaves). The acquired images may be calibrated by removing dark valuestherefrom and/or normalized and/or subjected to a smoothing process toeliminate noise (e.g., via median filtering). Spectral feature valuesmay be calculated which correspond to the known materials (or a knownclass of materials). For instance, a first value corresponding to a meanvalue of reflectance intensity may be calculated and associated with ayellow foam and a second value corresponding to a mean value ofreflectance intensity may be calculated and associated with tobaccoleaves. Spectral feature values may be acquired and stored within, e.g.,a non-transitory computer-readable storage medium which may be accessedby system components (e.g., processing devices, etc.) implemented incarrying out a method for detecting and/or removing foreign and/orproduct matter from a product stream. Based upon the spectral featurevalues (e.g., mean reflectance intensity) of the known materials, athreshold for each known material or class of known materials may becalculated and, e.g., similarly stored within a computer-readablestorage medium. The calculated threshold associated with each knownmaterial may be stored in a database in connection with the knownmaterial and subsequently accessed for implementation in a detectionprocess such as those described herein.

Systems useful in connection with the methods of detection and removaldescribed herein, may comprise the following components: an illuminationsource configured, e.g., to illuminate a portion of the product withlight spanning a wavelength range within or comprising a near-infraredor shortwave infrared wavelength range; a hyperspectral imaginginstrument configured to scan a line of the illuminated product andacquire a hyperspectral image of the line (i.e., a single row ofpixels); and a processor configured to: process the hyperspectral imageof the scanned line to obtain spectrum data for the image of the scannedline and/or one or more pixels of the image of the scanned line, analyzethe obtained spectrum data (e.g., on a line-by-line basis, on the basisof one or several pixels of a line, etc.) to determine whether theobtained spectrum data is inconsistent with predetermined spectrum datafor the product, and determine, when the spectrum data is inconsistentwith the predetermined spectrum data for the product, that the scannedline of product comprises foreign matter.

In some embodiments, the processor may further be configured tocalibrate the scanned image by removing dark values of the scannedimage, and/or normalize the scanned image and/or smooth the scannedimages to eliminate noise using, e.g., median filtering.

In some embodiments, the system may further comprise a foreign matterand/or product removal or separation device. In some embodiments, theprocessor may be further configured to generate an activation signal inresponse to a determination that a scanned line of product comprisesforeign matter wherein the processor can transmit the activation signalto a removal device. In response to the activation signal, the removaldevice may be configured to separate or remove some or all of theproduct and/or foreign matter corresponding to the scanned line in whichthe foreign matter was detected. In some embodiments, the method mayinvolve removal of the foreign material by a vacuum device and/or aforced fluid stream.

In some embodiments, the processor may, instead or in addition togenerating and transmitting an activation signal, generate a display(e.g., of a graph representing values associated with the scanned lineor a pixel of the scanned line, an image of the scanned line which maybe enhanced to indicate materials displayed within the image, etc.) forconveying the determination to a user (e.g., via a user interface),create a model of the image of a scanned line, print results associatedwith the determination, etc.

The methods and systems disclosed herein offer certain advantages overknown methods of detecting and analyzing products via hyperspectralimaging systems. For instance, embodiments disclosed herein do notrequire the implementation of end member extraction algorithms (e.g., tobuild spectral profiles, analyze scanned image data, etc.), the creationof a spectral library storing, e.g., full spectrum fingerprint data, theacquisition of dark and white references, etc. Thus, systems and methodsdisclosed herein offer significant computational savings overpre-existing hyperspectral imaging systems, enable methods to beperformed more quickly (e.g., systems may be operated with high conveyorbelt speeds), and may not require significant input energy orcomputational devices (e.g., a super computer), in contrast to knownsystems of hyperspectral imaging detection. Additionally, systems andmethods disclosed herein may be performed in real-time, as image data isacquired on a line-by-line basis. Methods and systems disclosed hereinmay be operable to detect and remove foreign matter in a more efficientand quick manner than past methods of foreign matter detection. Systemsand methods disclosed herein do not require the building of an imagecube from multiple scanned lines or the storage of multiple images,thus, providing storage savings, in addition to processing resourcesavings, over known detection systems implementing hyperspectral imagingdevices. For instance, an image acquired by a hyperspectral imagingdevice or camera may be thought of as comprising both spatial dimensions(e.g., M×N pixels) as well as a spectral dimension (e.g., comprised of anumber of spectral bands, B). Thus, where a line image is acquired andanalyzed separately from other lines acquired by the systems and methodsdisclosed herein, a width M of such an image may be assigned a value of1.

A schematic representing a basic hyperspectral imaging system which maybe implemented for scanning agricultural product P is depicted inFIG. 1. The system 100 includes at least one light source 102 forilluminating agricultural product P. Agricultural product P ispositioned on platform 104, which may be a fixed platform or a movingplatform (e.g., a conveyer belt). As shown, the at least one lightsource 102 may be mounted on an arm 106 for positioning the at least onelight source 102 relative to agricultural product P. Arm 106 may bemounted to frame 108 and may be fixed thereto or may be adjustable.Frame 108 may be moveable (e.g., so as to move over a portion of productP when the product is stationary) or may be a fixed structure (e.g., awall or fixture). Light source 102 may be arranged in any of a number ofpositions within system 100, fixed or adjustable, so long as lightsource 102 is capable of illuminating product P. In some embodiments,additional light sources may be provided (e.g., second light source110), which may be, e.g., fixed to frame 108. Alternatively, secondlight source 110 may be arranged in any number of positions withinsystem 100, fixed or adjustable.

In some embodiments, light source 102 may be configured to provide abeam of light of different wavelengths, such as those in the visiblelight range, infrared range, ultraviolet range, or spanning one or morethereof. Light source 102 may be selected, e.g., according to apreselected wavelength range. Light source 102 may be, e.g., a tungstenlight source, a xenon light source, a halogen light source, a mercurylight source, a UV light source, and/or a combination thereof. In someembodiments, light source 102 may be a broad band light source veeringthe visible as well as the infrared spectral range, such as a tungstenhalogen focused line light source.

In some embodiments, light source 102 may produce light within thenear-infrared range, spanning a wavelength range from at least about 900nm to about 2500 nm. In some embodiments, light source 102 may beconfigured to produce light spanning a wavelength range from at leastabout 900 nm to about 1700 nm. If desired, light source 102 may befiltered to reduce incident energy on scanned product. In embodimentshaving more than one light source, additional light sources, such assecond light source 110 may be similar to or distinct from light source102 (e.g., each light source may be of the tungsten halogen type; one ofthe light sources may be a halogen light whereas another may be atungsten type; one light source may be fixed and another may beadjustable; the light sources may emit light of similar or distinctwavelengths; one or more light sources may emit light in a nearinfra-red range whereas an additional light source may emit light in theultraviolet range, etc.). The number of light sources and type of lightsources implemented in system 100 can be selected so as to provide thedesired illumination energy.

In certain applications, the at least one light source 102 and/or thesecond light source 110 and/or additional light sources (not shown) maybe positioned to minimize the angle of incidence of a beam of light withrespect to agricultural product P. In some embodiments, system 100 orportions thereof may be enclosed within a structure (not shown) toprovide a dark-room-type environment. In some embodiments, such as thoseimplementing a near-infrared camera, fluorescent ambient light (e.g.,visible light) will not affect the data collected by system 100. In someembodiments a line light source may be arranged so that two line lightsare positioned along two sides of a product P or product stream provideillumination of a region of interest of product P or the product stream.In some embodiments, a light source may be positioned above and/or belowthe moving product stream.

Light source 102 and/or additional light sources, such as second lightsource 110 may illuminate only a region of product P to be scanned by ahyperspectral imaging device 112; however, in some embodiments, a largerregion than that scanned by imaging device 112 may be illuminated bylight source 102 and/or light source 110. In some embodiments lightsource 102 and/or additional light sources may be pulsed light sources.In some embodiments, light source 102 and/or additional light sourcesmay be continuous light sources. In some embodiments one or moreadditional hyperspectral imaging devices (or other types of imagingdevices) may be implemented to scan a same or different portion ofproduct P as that scanned by imaging device 112.

In preferred embodiments, light source 102 and any additional sources oflight implemented in system 100 are arranged so that the region to bescanned by hyperspectral imaging device 112 is uniformly illuminated andthe image acquired by imaging device 112 is not saturated.

Hyperspectral imaging device 112 may be a commercially availablehyperspectral imaging camera, such as those produced by HeadwallPhotonics of Fitchburg, Mass., those obtainable from Surface OpticsCorporation of San Diego, Calif., or other companies, or may be custombuilt according to the needs of system 100. In a preferred embodiment,hyperspectral imaging device 112 is a near-infrared imaging device or ashort wave infrared imaging device operating in a wavelength range of900-1700 nm or 900-2500 nm, respectively.

In some embodiments, hyperspectral imaging device 112 may be configuredto move relative to stationary product P. In preferred embodiments,product P may be positioned on movable platform 104 (e.g., a conveyorbelt), and may move relative to hyperspectral imaging device 112 (e.g.,conveyed past hyperspectral imaging device 112). In preferredembodiments, hyperspectral imaging device 112 is a line-scanhyperspectral imaging device, operable to acquire image data for ascanned line of a product stream of product P. In some embodiments,imaging device 112 may acquire data representative of a scanned line ofproduct oriented across platform 104.

Hyperspectral imaging device 112 may include and/or be operably incommunication with a processor 114. Processor 114 may be configured toreceive scanned image data from imaging device 112 and process thereceived image data. In preferred embodiments the scanned image data,acquired by imaging device 112 and received by processor 114, compriseswavelength bands in the 900-1700 nm or 900-2500 nm wavelength range.Processor 114 may be configured to process received band data byemploying one or more unique algorithms to determine the nature of thescanned product stream.

For instance, hyperspectral imaging device 112 may acquire hyperspectralimage data of an illuminated line of a tobacco product stream. Thetobacco product stream may be illuminated by light source 102 and/orsecond light source 110 operating within, e.g., the near-infraredwavelength range. Processor 114 may receive the hyperspectral image datafrom the hyperspectral imaging device 112 and may process the receivedimage data according to, e.g., a predetermined algorithm. Thehyperspectral image data may be processed by processor 114 on apixel-by-pixel basis as the line of the product stream is scanned byimaging device 112. Thus, spectrum data corresponding to a pixel of thescanned line of the product stream may be determined. Alternatively oradditionally, spectrum data corresponding to the scanned line may beacquired. Processor 114 may be configured to compare the spectrum datafor the pixel and/or scanned line to a predetermined threshold valueand, on the basis of such a comparison, determine whether the spectrumdata for the pixel and/or scanned line is representative of tobacco orforeign (e.g., non-tobacco) matter. Hyperspectral imaging device 112 andprocessor 114 may be configured to obtain and process multiple scannedlines of a product stream on a line-by-line basis in real-time. That is,in embodiments of system 100, processor 114 is configured to detectforeign matter within a product stream without constructing an imagecube of multiple scanned lines of the product stream.

Processor 114 may be in operable communication with a controller orcontrol system via user interface 116. Interface 116 may include, e.g.,a display and input device to enable a user to control system 100 or oneor more components thereof. For instance, a user may provide data viainterface 116 to processor 114 to be utilized in a determination as toacceptable product stream matter and unacceptable product stream matter.

In systems in which removal of scanned product is desired, processor 114may be in operable communication with a product (or foreign matter)removal device 118. In systems wherein platform 104 is a movableplatform, removal device 118 may receive a signal in response to aprocess performed by the processor 114 (e.g., analysis of ahyperspectral image of product stream) and may operate to remove some orall of a given portion of product stream based upon the received signal.Removal device 118 may remove product or non-product material by, e.g.,vacuum or suction, physical removal via robotic arms, ejecting a fluidstream, etc. Removal device 118 may be positioned relative to productstream moving on platform 104 depending upon the type of removalmechanism employed. The activation mechanism for removal device 118(e.g., a signal provided by processor 114 in response to analysis ofacquired hyperspectral image data) may be based upon, e.g., the speed ofmovable platform 104 and distance between the position of the imagingdevice 112 and/or the position of product stream which is scanned byimaging device 112 and the removal device 118 and/or the position atwhich product or foreign matter is removed from the product stream byremoval device 118.

To ensure accurate scanning, analysis, and/or removal of product stream,in the case of a movable platform 104, the platform (e.g., conveyorbelt) speed may be tracked by, e.g., a sensor or triggering mechanism(e.g., a shaft encoder). The speed of the belt may be communicated tothe processor 114 for use in various processes performed by system 100.

Referring now to FIG. 2, one possible embodiment of a detection andseparation system 200, as disclosed herein, is shown schematically.Detection and separation system 200 may implement one or more componentsof system 100 (e.g., hyperspectral imaging device 112 may correspond to216; light source 102 may correspond to 208; etc.). In operation, anagricultural product stream 202, which may be a tobacco stream,containing foreign material, such as foil, cellophane, warehouse tags,cardboard, foam, paper, or oil or other lubricant containing material,is delivered from a processing line by a conveyor 204. Conveyor 204 maybe, e.g., a vibrating inclined conveyor which vibrates as shown byarrows v. In some embodiments, conveyor 204 ends above another conveyor206, which can be, e.g., an ordinary conveyor belt. Conveyor 204 may bespaced vertically above conveyor 206 a sufficient distance toaccommodate the remainder of system 200 as described herein. As productstream 202 reaches the end of conveyor 204, it drops under the influenceof gravity in a cascade C to conveyor 106. In one embodiment, becauseconveyor 204 is inclined, the product stream does not have as great ahorizontal velocity when it falls, so that cascade C does not have anysignificant front-to-back horizontal spread.

In alternative embodiments, detection and separation system 200 mayinclude a single conveyor 204 for inspecting finished product, such ascigarettes, smokeless tobacco containers, SNUS pouches, etc., withproduct rejection, described in more detail below, taking place on thesame conveyor. In such embodiments, conveyor 204 may be positionedhorizontally or at an incline.

The detection and separation system 200 includes at least one lightsource 208 for providing a beam of light. As shown, the at least onelight source 208 may be mounted on an arm 210 for positioning the atleast one light source 208 in proximity to the product stream 202. In anembodiment, arm 210 may be mounted to structure 212 which may be, e.g.,a cabinet, and arm 210 may be either fixed thereto or moveablyposition-able, as will be described herein. Additional light sources(not shown) may also be provided and mounted to the structure 212 or,optionally, to one or more additional arms (not shown), which in turnmay be mounted to structure 212.

As shown in FIG. 2, the light from the at least one light source 208 maybe directed toward the cascade C of the product stream 202 by the mirror214. In alternative embodiments, light source 208 and/or additionallight sources (not shown) may be positioned so as to directly illuminateproduct stream. In some embodiments, the light sources of system 200 maybe positioned to minimize scattering effects. A hyperspectral image ofproduct stream 202 may be acquired using hyperspectral imaging camera216, which, in some embodiments, may receive light reflected by mirror218. In alternative embodiments, the hyperspectral imaging camera 216may receive light directly reflected from product stream 202.

In some embodiments, the at least one light source 208 and/or theadditional light sources (not shown) may be positioned to minimize theangle of incidence of a beam of light with the agricultural productstream. In order to segregate ambient light form the light provided bysystem 200, walls (not shown) may be added to form an enclosure toprovide a dark-room-like environment for a system 200.

A computer processing device may be included within system 200. Forinstance, a processor 220 may be mounted within structure 212. Processor220 may be capable of rapidly handling system data and may beprogrammed, e.g., to compare received image data with threshold valuesas discussed above in connection with FIG. 1 and as further discussedherein. Processor 220 may be constituted by one or more conventionalprocessors. Processor 220 operates to execute computer-readable steps,contained in program instructions, so as to control, e.g., variouscomponents of system 200 and/or system 200 as a whole. A device forproviding uninterrupted power to processor 220 may be provided and,e.g., mounted within structure 212. A regulated power supply (not shown)may be provided to ensure that a tightly controlled source of power issupplied to system 200.

In some embodiments, system 200 is provided with a user interface 222which enables an operator (not shown) to observe and/or control variousoperational aspects of system 200. User interface 222 is operablyconnected to processor 220 (e.g., via a networked computer system,etc.). User interface 222 may include a CRT or LCD panel for outputdisplay. User interface 222 may include a keyboard, touch-screen, mouse,joystick, or other known input means operable to enable an operator toinput data into and/or control various aspects of system 200. Userinterface 222 may be located in close proximity to processor 220 and theproduct stream being analyzed within system 200 or may be situated at aremote location. Via user interface 222, an operator may, e.g., viewrepresentations of scanned portions of the product stream 202, e.g., inreal-time as such portions are being processed and analyzed by system200 or subsequent to scanning and analysis. In some embodiments, anoperator may determine between acceptable product and unacceptableproduct (e.g., in real-time, prior to processing via configuring theprocessor 220, etc.) via user interface 222 (e.g., by viewing displayeddata produced by processor 220 in response to scanning and analysis ofproduct stream 202). In some embodiments, the determination betweenacceptable product and unacceptable product, the removal thereof, etc.is carried out via the processor 220 without operator input (e.g.,automated).

In embodiments, when system 200 detects foreign material in productstream 202, processor 220 may produce a signal which may be communicatedto ejector manifold 224, which is operably connected to processor 220and which is positioned in downstream relation to the region illuminatedby the at least one light 208. Ejector manifold 224 is in fluidtransmission relation to the trajectory of the product stream 202. Theejector manifold 224 may include one or several ejector nozzles 226,which may be individually directed and controlled to selectively removeundesirable product material 228 from the product stream 202. Theejector nozzles 226 act as conduits for directing fluid pulses todislodge or otherwise re-direct product material traveling in thetrajectory of product stream 202. Individual ejector nozzles 226contained in the ejector manifold 224 may be driven by a plurality ofremoval signals, which may be provided by processor 220, an operator viauser interface 222, or via some other control system (not shown) inoperable communication with ejector manifold 224.

In some embodiments, ejector nozzles 226 may be connected to a source ofhigh pressure fluid. In some embodiments, the fluid is air atapproximately 80 psi. In other embodiments, the fluid may be a gas orliquid other than air (e.g., steam, water, etc.). When one of theejector nozzles 226 opens in response to a signal, a blast of air A isdirected against that portion of cascade C in which the foreign materialhas been detected (e.g., by processor 220 via hyperspectral imaging dataacquired by camera 216) to force that portion 228 of the product stream202 and/or foreign material to fall into receptacle 230 for additionalprocessing if necessary (e.g., manual sorting, removal by a conveyor toanother system for processing, etc.). In the case usable product isforced into receptacle 230, it may be returned to the product processingline upstream or downstream of system 200, depending on whether or notrescanning of said usable product is desired.

As may be appreciated, system 200 may enable tobacco or other materialsto be processed at greater rates than a system in which the tobacco orother materials are scanned on a standard belt conveyor. This is becausewhen tobacco or other product material is optically scanned on a belt,it is desirable that the tobacco be in or close to a “monolayer” orsingle layer of particles, for all of the particles on the belt to bevisible to the hyperspectral imaging camera 216. In embodiments ofsystem 200, when the tobacco or other material falls in cascade C,relative vertical motion between the various particles of tobacco andforeign material is induced by the turbulence of the falling stream, sothere is a greater probability that a particular piece of foreignmaterial will be visible to hyperspectral imaging camera 216 at somepoint in its fall. Relative vertical motion also results if the foreignmaterial is significantly lighter or heavier than the tobacco so that ithas a greater or lesser air resistance as it falls. Relative verticalmotion may be enhanced by the vibration of conveyor 204 which bringslighter material to the surface of the tobacco before it falls incascade C, making the lighter material, which may represent foreignmaterial, easier to detect, as in a monolayer. Alternative systems tothe cascading system may be used in situations where a “monolayer” orsimilar type of layer is desired prior to scanning the product streamsuch as those which use brushes to separate product stream matter, etc.

The inclination of conveyor 204, in reducing the horizontal spread ofcascade C as discussed above, also enhances relative vertical motionbecause the particles in cascade C have little or no horizontal velocitycomponent. Any horizontal velocity component that a particle has when itfalls off conveyor 204 is small because conveyor 204 is inclined, andair resistance quickly reduces the horizontal motion to near zero. Therelative vertical motion allows a relatively thicker layer of tobacco orother material to be scanned, so that a greater volume can be scannedper unit of scanning area. Given a constant rate of area scanned perunit time, the increased volume scanned per unit area translates into ahigher volume of tobacco or other material scanned per unit time.

In operation, systems for detecting, analyzing and/or removing foreignmatter (i.e., “non-product related materials” or “NPRM”) from a productstream, such as those described in connection with systems 100 and 200,may operate by a process 300 such as that set forth in the flow chart ofFIG. 3.

In step 302 of method 300, a line (e.g., of moving tobacco productstream intermixed with non-product related material) may be illuminated(e.g., by one or more light sources such as those discussed inconnection with FIG. 1 and FIG. 2, above). The illuminated line may beilluminated with near-infrared light spanning at least from about 900 nmto about 2500 nm. In some embodiments, the line may be illuminated withat least near-infrared light spanning from about 900 nm to about 1700nm.

In step 304, hyperspectral image data (e.g., a hyperspectral reflectanceimage) may be acquired. Hyperspectral imaging data may be obtained byhyperspectral imaging devices, such as device 112 of system 100 ordevice 216 of system 200. In embodiments, the imaging data may beobtained by a line-scan hyperspectral imaging instrument. In preferredembodiments, hyperspectral imaging data is obtained by a hyperspectralimaging instrument operating within the shortwave infrared (SWIR) ornear infra-red (NIR) range. For instance the SWIR or NIRMicro-Hyperspec® Hyperspectral Sensors produced by Headwall Photonics,of Fitchburg, Mass. could be implemented in connection with method 300.In preferred embodiments, the image data acquired is single row ofpixels that represents a line of the scanned product stream.

In step 306, obtained images may be calibrated (e.g., by a processingdevice) to remove dark values and normalize the images). If desired,calibration coefficients may be applied to compensate for fluctuationsin operating conditions (e.g., light intensity, ambient conditions,etc.).

In step 308, obtained images may be smoothed (e.g., by a processingdevice) to eliminate noise. Smoothing of the images may be accomplishedby, e.g., a median filtering process. Such as one represented by thefollowing equation: (y[m,n]=median {x [i, j], (i, j)εw), where wrepresents a neighborhood defined by, e.g., a user, centered aroundlocation [m,n] in the image.

In step 310, a value may be associated with one or more pixels of thescanned image data (e.g., with a pixel of an image representing ascanned line of product stream). Such a value may be calculated basedupon a calculated Euclidean distance for each pixel. The Euclideandistance may be calculated by the following formula: d(p,q)=√{squareroot over (Σ_(i=1) ^(n)(q_(i)−p_(i))²)}, wherein p=(p₁, p₂ . . . ,p_(n)) and q=(q₁, q₂ . . . , q_(n)) and p and q are two points inEuclidean n-space. Each pixel of a scanned line may be associated with acorresponding calculated value. Alternatively, other classificationtechniques may be used such as Mahalanobis distance measure,Bhattacharya distance measure, spectral angle distance measure (SAM),maximum entropy classifier, curve fitting based on regression analysis,support vector machine (SVM), clustering techniques including k-meansclustering, nearest neighbor clustering, or techniques based on neuralnetwork classifiers. In an embodiment, a combination of various datareduction and analysis techniques like principal component analysis,orthogonal decomposition, spectral mixture resolution, etc. could beused as a first step to reduce the dimensionality of the data andextract relevant features which could be used by a classifier.

In step 312, for each pixel associated with a corresponding calculatedvalue, a determination may be made as to whether the value is greaterthan or less than a pre-determined threshold. The value (compared to thepre-determined threshold) for each pixel may be the Euclidean distancefor that pixel, as described in step 310. In some embodiments, thescanned line may be associated with a corresponding calculated value andmay be compared to a pre-determined threshold. In some embodimentsvalues may be calculated and implemented within method 300 which arerepresentative of multiple pixels or a scanned line as a whole ratherthan or in addition to values of individual pixels of a scanned line.

In step 314, the image of the line may be analyzed to determine whetherthe line contains foreign matter (e.g., non-tobacco related material).For instance, if the Euclidean distance associated with a pixel of theline is greater than the pre-determined threshold, a determination maybe made that the pixel corresponds to foreign matter (i.e., contaminantor non-product related material). If the Euclidean distance associatedwith a pixel of the line is less than the threshold value, adetermination may be made that the corresponding scanned matter isproduct material (e.g., uncontaminated tobacco). Alternatively, thepre-determined threshold may be selected so that, if the Euclideandistance associated with a pixel of the line is greater than thepre-determined threshold, the pixel may be determined to correspond toproduct matter, whereas a Euclidean distance associated with a pixelwhich is less than the pre-determined threshold may correspond toforeign matter. The threshold value may represent, e.g., a calculateddata point, a distance from a calculated data point within which amaterial must fall to be considered similar to a material associatedwith a material corresponding to the data point, etc.

Steps 302 thru 314 may be carried out by various components of thesystems 100 and 200, including but not limited to the processors, lightsources and hyperspectral imaging devices disclosed therein, and othersystems and system components similar thereto.

The pre-determined threshold value described in connection with method300 may vary depending upon the application of method 300. Method 400depicted in FIG. 4 provides one manner in which such a pre-determinedthreshold value may be calculated.

In step 402, known material (e.g., tobacco or contaminant material) maybe illuminated with, e.g., near-infrared light spanning at least fromabout 900 nm to 2500 nm. In some embodiments, the material may beilluminated with light spanning at least 900 nm to 1700 nm. Inalternative embodiments, other electromagnetic wavelength ranges (e.g.,ultraviolet, visible light, infrared, etc.) may be implemented inmethods 300 and/or 400 instead of or in addition to those in thenear-infrared or shortwave infrared wavelength ranges.

In step 404, hyperspectral reflectance image data of the known material(e.g., product material or non-product related material/foreign matter)may be acquired with a hyperspectral imaging device, such as an NIRimaging device. The known material imaging data may be acquiredseparately from that acquired for other known materials. In someembodiments, the hyperspectral reflectance imaging device may be similarto or the same imaging device as that used to acquire image data of theproduct stream in method 300.

In step 406, the acquired images may be calibrated, e.g., by removingdark values and normalizing the image. In step 408, the images may besmoothed to reduce or eliminate noise, using a process such as medianfiltering (similar to that described in connection with step 308 ofmethod 300).

In step 410, spectral feature values, e.g., a mean value of reflectanceintensity, may be calculated for each known material and/or for a classof known material, for which image data is acquired. In step 412,threshold values may be determined for known materials and/or productmaterial. Threshold values may be implemented in processes similar tothose described herein (e.g., in method 300) and may be compared toscanned line or pixel data acquired by a real-time detection and removalsystem such as systems similar to those described herein.

Threshold values associated with known materials may be stored, e.g., ina non-transitory computer readable medium accessible by, e.g., systemssuch as those disclosed in FIG. 1 and FIG. 2, for implementation in adetection and/or removal process.

In FIG. 5, sample spectra from an NIR camera used in connection withmethods and systems disclosed herein is presented in the form of agraph. The graph represents data collected with an NIR camera at awavelength range of 900 nm to 1700 nm.

The various lines of the graph represent spectra associated with knownmaterials. The line corresponding to 502 represents “white foam.” Theline corresponding to 504 represents spectrum data associated with a“belt” (e.g., such as a movable conveyor belt). The line correspondingto “tobacco” is labeled 506. The line corresponding to 508 representsspectrum data associated with “yellow foam.” The line corresponding to510 represents spectrum data corresponding to a “white reference.”

As may be appreciated by one skilled in the art, yellow and white foamare known contaminants present in tobacco product streams. As can beunderstood from the spectra data presented in FIG. 5, the spectralsignatures obtained by an NIR camera operating at a wavelength range of900 nm to 1700 nm are extremely stable and almost rectangular throughoutthat wavelength range. The pronounced spectral difference illustratedbetween the known contaminants (e.g., 502 and 508) and the tobacco (506)enables methods and systems disclosed herein for detecting foreignmatter in a product stream to be carried out accurately.

FIG. 6 provides an example of results that may be produced by systemsand methods described herein. As can be seen, a conveyor belt may conveyproduct material (e.g., tobacco) along with contaminants/foreign matter(e.g., foam). Using methods and systems disclosed herein, the foam maybe detected within the conveyed product stream. The foam may be detectedby analyzing a hyperspectral image acquired of a scanned line of productto determine whether individual pixels (e.g., representative of 0.25inches of contaminant material) should be identified as product orcontaminant. The analysis may implement algorithms discussed in moredetail herein. The analysis may result in the identification of only thecontaminant material present in a scanned line of product. In someembodiments, results may be displayed in a manner similar to that ofFIG. 6 (e.g., via a user interface). Where contaminant material isidentified within a scanned line, as discussed herein, a signal may begenerated and provided to a contaminant removal device which may removecontaminant material (or a line of product containing the contaminantmaterial) from the product stream. This process may be carried out inreal-time (e.g., without slowing the conveyor belt to remove thecontaminated product material, etc.).

While the foregoing describes in detail systems and methods of detectingforeign material in product, or detecting and removing foreign materialfrom product, with reference to specific embodiments thereof, it will beapparent to one skilled in the art that various changes andmodifications and equivalents to the systems and methods disclosedherein may be employed, which do not materially depart from the spiritand scope of the invention.

We claim:
 1. A method of detecting foreign material within an agricultural product stream in real-time, the method comprising: illuminating a portion of the agricultural product stream with light spanning a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; scanning a line of the illuminated agricultural product stream to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; processing the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line; and comparing the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the agricultural product stream.
 2. The method of claim 1, wherein the scanned line of agricultural product stream is perpendicular to a direction in which the agricultural product stream is being moved.
 3. The method of claim 1, wherein the predetermined spectrum data is a calculated value based upon spectral data associated with one of the following materials: tobacco, foam, cardboard, plastic, foil, lubricant, or paper.
 4. The method of claim 1, further comprising a step of: determining, based upon the comparing step, the location of foreign material in the product stream; or determining, based upon the comparing step, that the scanned line of the agricultural product stream does not contain foreign material.
 5. The method of claim 1, wherein the processing step comprises calculating the Euclidean distance associated with one or more pixels of the hyperspectral image of the scanned line, wherein the Euclidean distance is calculated by the formula: d(p,q)=√{square root over (Σ_(i=1) ^(n)(q_(i)−p_(i))²)}, wherein p=(p₁, p₂ . . . , p_(n)) and q=(q₁, q₂ . . . , q_(n)) and p and q are two points in Euclidean n-space.
 6. The method of claim 1, wherein the scanning step includes sensing a spectrum of light reflected, scattered, or emitted from the agricultural product stream with at least one sensor; wherein: (a) the scanning is performed by a line-scan hyperspectral imaging device that includes the at least one sensor; (b) the at least one sensor is an Indium Gallium Arsenide (InGaAs) sensor; and/or (c) the at least one sensor is operable to sense light in a range of about 900 nm to 1700 nm or about 900 nm to 2500 nm.
 7. The method of claim 1, wherein: (a) the scanning is performed by a line-scan hyperspectral imaging device wherein the hyperspectral imaging device operates in the range of about 900 nm to 1700 nm or about 900 nm to 2500 nm; and/or (b) the illuminating is performed by a tungsten halogen light source.
 8. The method of claim 1, wherein the agricultural product stream is a continuously moving agricultural product stream and is conveyed on a conveyer belt operating at a speed of about 80-100 ft/min.
 9. The method of claim 1, wherein multiple lines of agricultural product stream are individually scanned consecutively, and the consecutively scanned lines are separately processed in-real time.
 10. The method of claim 9, wherein the consecutively scanned lines are processed without constructing a hyperspectral image cube of the multiple lines.
 11. The method of claim 1, wherein the processing step comprises: calibrating the hyperspectral image of the scanned line by removing dark values of the hyperspectral image; normalizing the hyperspectral image of the scanned line; and smoothing the hyperspectral image of the scanned line to eliminate noise using median filtering.
 12. The method of claim 1, wherein the predetermined spectrum data is a threshold value associated with a single product material, a class of product materials, a single known foreign material, or a class of known foreign materials.
 13. The method of claim 12, wherein the threshold value is determined by a method comprising: illuminating the product material, a class of product materials, a known foreign material, and/or a class of known foreign materials with light spanning a wavelength range including near-infrared and/or shortwave infrared wavelengths; separately scanning the product material, the class of product materials, the known foreign material, and/or the class of known foreign materials to acquire hyperspectral reflectance image data for at least one of the respective product material, the class of product materials, the known foreign material, or the class of known foreign materials; calculating, for at least one of the respective product material, the class of product materials, the known foreign material, or the class of known foreign materials, a mean value of reflectance intensity; and based upon the one or more calculated mean values of reflectance intensity, determining the threshold value associated with at least one of the respective product material, the class of product materials, the known foreign material, or the class of known foreign materials.
 14. The method of claim 1 further comprising: generating an activation signal in response to determining that the scanned line of agricultural product stream comprises foreign material; transmitting the activation signal to a removal system; and removing the foreign material from the scanned line.
 15. The method of claim 14, wherein the foreign material is removed by a vacuum device and/or a forced fluid stream.
 16. The method of claim 9, further comprising: separately analyzing each consecutively scanned line to determine, for each scanned line, whether foreign material is detected; and mapping the foreign material when foreign material is detected in two or more consecutively scanned lines.
 17. The method of claim 9, further comprising: separately analyzing each consecutively scanned line to determine, for each scanned line, whether foreign material is detected; and generating a signal indicative of a false positive when foreign matter is detected in only a single scanned line.
 18. The method of claim 1, wherein: (a) the method is performed without the implementation of end member extraction algorithms so as to reduce processing time; (b) the predetermined spectrum data does not include full spectrum fingerprint data so as to reduce storage requirements of a system implementing the method; and/or (c) the method does not require the acquisition of dark and white references so as to reduce storage requirements of a system implementing the method.
 19. A system for detecting foreign material within an agricultural product stream in real-time, the system comprising: an illumination device configured to illuminate a portion of the agricultural product stream at a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; a hyperspectral imaging device configured to scan a line of the illuminated agricultural product stream to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; and a processor configured to: process the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line, and compare the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the agricultural product stream.
 20. The system of claim 19, wherein the processor is further configured to: determine, based upon the compared spectrum data, the location of foreign material in the product stream; or determine, based upon the compared spectrum data, that the scanned line of the agricultural product stream does not contain foreign material.
 21. A non-transitory computer readable medium having instructions stored thereon, the instructions being configured to cause a processor to execute the steps of the method of claim
 1. 22. A method of detecting foreign material within stationary agricultural product in real-time, the method comprising: illuminating a portion of the stationary agricultural product with light spanning a wavelength range including or within near-infrared and/or shortwave infrared wavelengths; moving a hyperspectral imaging device over a portion of the stationary agricultural product while simultaneously scanning a line of the illuminated stationary agricultural product to acquire a hyperspectral image of the line, the hyperspectral image of the line having a width of a single pixel; processing the hyperspectral image of the scanned line to obtain spectrum data for one or more pixels of the hyperspectral image of the scanned line; and comparing the obtained spectrum data of the one or more pixels to predetermined spectrum data to determine whether the obtained spectrum data is indicative of foreign material within the scanned line of the stationary agricultural product. 